Video transcoding system with quality readjustment based on high scene cost detection and method for use therewith

ABSTRACT

A system for transcoding a video signal into a transcoded video signal, includes a high scene cost detection module that detects a high scene cost corresponding to at least one image of the video signal. An encoder section generates the transcoded video signal, wherein, when the high scene cost is detected, an enhanced quality is assigned to at least one central region.

CROSS REFERENCE TO RELATED PATENTS

None

TECHNICAL FIELD OF THE INVENTION

The present invention relates to video transcoding used in devices suchas video processing devices.

DESCRIPTION OF RELATED ART

Video encoding has become an important issue for modern video processingdevices. Robust encoding algorithms allow video signals to betransmitted with reduced bandwidth and stored in less memory. Standardshave been promulgated for many encoding methods including the H.264standard that is also referred to as MPEG-4, part 10 or Advanced VideoCoding, (AVC). While standards set forth many powerful techniques,further improvements are possible in the performance and the speed ofimplementation of such methods. Further, the promulgation of multipleformats has led to circumstances where video signals in one format needto be transcoded in another format for use on a particular device, fortransmission or for storage. The accuracy of these transcoding methodsface the scrutiny of users that are becoming accustomed to higherresolution and better picture quality.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of ordinary skill in the artthrough comparison of such systems with the present invention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 presents a block diagram representation of a video processingdevice 125 in accordance with an embodiment of the present invention.

FIG. 2 presents a block diagram representation of a video transcodingsystem 102 in accordance with an embodiment of the present invention.

FIG. 3 presents a pictorial representation of an image 300 in accordancewith an embodiment of the present invention.

FIG. 4 presents a block diagram representation of a video distributionsystem 175 in accordance with an embodiment of the present invention.

FIG. 5 presents a block diagram representation of a video storage system179 in accordance with an embodiment of the present invention.

FIG. 6 presents a flowchart representation of a method in accordancewith an embodiment of the present invention.

FIG. 7 presents a flowchart representation of a method in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION INCLUDING THE PRESENTLY PREFERREDEMBODIMENTS

FIG. 1 presents a block diagram representation of a video processingdevice 125 in accordance with an embodiment of the present invention. Inparticular, video processing device 125 includes a receiving module 100,such as a set-top box, television receiver, personal computer, cabletelevision receiver, satellite broadcast receiver, broadband modem, 3Gtransceiver or other information receiver or transceiver that is capableof receiving a video signal 110 from one or more sources such as abroadcast cable system, a broadcast satellite system, the Internet, adigital video disc player, a digital video recorder, or other videosource. Video transcoding system 102 is coupled to the receiving module100 to transcode the video signals 110 to form transcoded video signal112.

In an embodiment of the present invention, the video signal 110 caninclude a broadcast video signal, such as a television signal, highdefinition television signal, enhanced high definition television signalor other broadcast video signal that has been transmitted over awireless medium, either directly or through one or more satellites orother relay stations or through a cable network, optical network orother transmission network. In addition, the video signal 110 can begenerated from a stored video file, played back from a recording mediumsuch as a magnetic tape, magnetic disk or optical disk, and can includea streaming video signal that is transmitted over a public or privatenetwork such as a local area network, wide area network, metropolitanarea network or the Internet.

Video signal 110 and transcoded video signal 112 can each be encoded inaccordance with a digital video format such as H.264, MPEG-4 Part 10Advanced Video Coding (AVC) or other digital format such as a MovingPicture Experts Group (MPEG) format (such as MPEG1, MPEG-2 or MPEG4),Quicktime format, Real Media format, Windows Media Video (WMV) or AudioVideo Interleave (AVI), or another digital video format, either standardor proprietary.

The video transcoding system 102 includes a high scene cost detectionmodule 150 that will be described in greater detail in conjunction withmany optional functions and features described in conjunction with FIGS.2 -7 that follow.

FIG. 2 presents a block diagram representation of a video transcodingsystem 102 in accordance with an embodiment of the present invention. Inparticular, video transcoding system 102 operates in accordance withmany of the functions and features of the H.264 standard, the MPEG-4standard, VC-1 (SMPTE standard 421M), MPEG-2 or other standard, totranscode video input signals 110 that are received via a signalinterface 198 into another digital video format and output as transcodedvideo signal 112.

The video transcoding system 102 includes an transcoder section 103,signal interface 198, processing module 230, and high scene costdetection module 150. The processing module 230 can be implemented usinga single processing device or a plurality of processing devices. Such aprocessing device may be a microprocessor, co-processors, amicro-controller, digital signal processor, microcomputer, centralprocessing unit, field programmable gate array, programmable logicdevice, state machine, logic circuitry, analog circuitry, digitalcircuitry, and/or any device that manipulates signals (analog and/ordigital) based on operational instructions that are stored in a memory,such as memory module 232. Memory module 232 may be a single memorydevice or a plurality of memory devices. Such a memory device caninclude a hard disk drive or other disk drive, read-only memory, randomaccess memory, volatile memory, non-volatile memory, static memory,dynamic memory, flash memory, cache memory, and/or any device thatstores digital information. Note that when the processing moduleimplements one or more of its functions via a state machine, analogcircuitry, digital circuitry, and/or logic circuitry, the memory storingthe corresponding operational instructions may be embedded within, orexternal to, the circuitry comprising the state machine, analogcircuitry, digital circuitry, and/or logic circuitry.

Processing module 230, and memory module 232 are coupled, via bus 250,to the signal interface 198 and a plurality of other modules, such ashigh scene cost detection module 150, decoding module 240 and encodingmodule 236. The modules of video transcoding system 102 can beimplemented in software, firmware or hardware, depending on theparticular implementation of processing module 230. It should also benoted that the software implementations of the present invention can bestored on a tangible storage medium such as a magnetic or optical disk,read-only memory or random access memory and also be produced as anarticle of manufacture. While a particular bus architecture is shown,alternative architectures using direct connectivity between one or moremodules and/or additional busses can likewise be implemented inaccordance with the present invention.

Transcoder section 103 can operate by decoding the video signal 110 viadecoding module 240 and re-encoding the decoded result via encodingmodule 236 to produce the transcoded video signal 112. However, decodingmodule 240 and encoding module 236 can alternatively cooperate tootherwise utilize parameters of the video signal 110 to transform thevideo signal 110 into a format of transcoded signal 112. In particularsimilarities between the format of the video signal 110 and thetranscoded video signal 112 can be leveraged to reduce the amount ofcomputation required and to avoid a full decoding and re-encoding.

In operation, high scene cost detection module 150 detects a high scenecost in an image, such as a frame or field of video signal 110.Transcoder section 103, when the high scene cost is detected, 103generates the transcoded video signal 112, with an enhanced qualityassigned to at least one central region. Consider an example where thevideo signal 110 has a scene like flowing and splashing water which canrequire many more bits to encode than the target bit rate, especially ifthe scenes lasts for more than a few seconds. Whether the transcodedvideo signal 112 is a variable bit rate (VBR) or constant bit rate (CBR)signal, the presence of such tough scenes can require special attentionto keep the bit rate from deviating too much from a desired or targetedbit rate. In particular, stringent bit allocation constraints couldforce a large quantization parameter in such scenes, resulting in poorvisual quality.

The present invention operates by detecting tough scenes based on highscene cost and by enhancing the quality to one or more central regionsof the screen to capitalize on a human visual model that places moreimportance on the center of the screen. Transcoded signal 112 isgenerated with more quality emphasis on the center portion of the image,as compared to the peripheral parts. In particular, the quality of acentral region of the image can be enhanced even if the tough portion ofthe image extends into the peripheral portions, outside the centralregion. By nature, the viewer focuses his or her attention to the middleof the image with enhanced quality. The result can be a transcoded videosignal 112 with greater perceived quality for the viewer.

In an embodiment of the present invention, the video signal 110 is anMPEG-2 signal. High scene cost detection module 150 can use encodingparameters such as MPEG-2 based statistics to evaluate the scene cost ofan upcoming scene and to detect a potential upcoming high scene cost.Observations derived from actual images indicate that difficult sceneswith fairly good quality in MPEG-2 often use many more bits, b, and havehigher variance values, var, for some or all of the macroblocks of theimage than easy scenes. Quantization step values (QP) also tend to berelatively higher for difficult or tough scenes. A scene cost, C, can bedetermined by combining these parameters effectively in the form of afunction,ƒ:f

C=η(b, var, QP)

A weighted linear function of these three parameters can be employed,with weighting coefficients chosen to normalize the particularparameters or can be otherwise be generated based on experimentalresults. Alternatively, a nonlinear function can likewise be employed.

In this example, the high scene cost module 150 operates to detect ahigh scene cost by comparing the calculated scene cost for an image,such as an MPEG-2 frame of the video signal 110, to a scene costthreshold. If the scene cost compares unfavorably to the scene costthreshold, such as when the scene cost exceeds the threshold, a highscene cost is detected. If the scene cost compares favorably to thescene cost threshold, such as when the scene cost does not exceed thethreshold, a high scene cost is not detected (an “easy” scene). Thescene cost threshold can also be determined experimentally based on theanalysis of actual scenes of differing complexities including somescenes predetermined to be “tough”.

Once a high scene cost is detected that corresponds to a particularimage, as discussed, the overall visual quality can be enhanced byenhancing the quality of one or more central regions of the image—whencompared with other areas outside the central region or regions. In anembodiment of the present invention, the transcoder section 103 adjuststhe values of QP to enhance the quality in the central region or regionscompared with the periphery of the image. For example, the QP can belowered in the central region differently for different macroblocksbased on the relative difference of the MPEG-2 macroblock QP with theaverage MPEG-2 QP for that frame. In this example, a QP Reduction factorfor the nth macroblock, QPR_(n), can be found based on:

QPR _(n)=η(MQP_(n)−MQP_(Av))

Where, MQP_(n) is the MPEG-2 based QP for the nth macroblock andMQP_(Av) is an average MPEG-2 QP for the entire image. QP in the areasoutside of the central region could be increased, in particular tomaintain CBR operation, resulting in diminished quality in theseregions. QP in the areas outside of the central region can leftunaltered, such is in VBR operation, still with quality that is lowerthan the enhanced quality of the central region.

In addition, the buffer fullness can be assessed before applying a QPredistribution to enhance visual quality in tough scenes. If the bufferfullness is within limits determined by comparing the buffer fullness toa buffer threshold B, the QP adjustments can be applied. If the bufferfullness compares unfavorably to the buffer threshold, the QPadjustments are not applied. Further, upper limits can optionally be setfor the macroblock QP based on the buffer fullness. This dynamicmodification of the QP helps maintain quality and also ensures bitrateconformance by taking into consideration buffer fullness. The mechanismof redistribution of QP and thereby bits can be very useful in toughscenes involving splashing or flowing water which invoke bitrateconstraints.

FIG. 3 presents a pictorial representation of an image 300 in accordancewith an embodiment of the present invention. In particular, image 300can be a frame or field of a video signal, such as video signal 11O. Aregion 302 is presented in the center of the frame 300. When a highscene cost is detected, the QP for macroblocks within the region 302would be lowered relatively as discussed in the various examplespresented in conjunction with FIG. 2. As discussed in conjunction withFIG. 2, the QP for macroblocks in the region 304, outside of the region302, could be unaltered or raised, depending on the implementation.While region 302 is shown as a single rectangular region, multipleregions of variable shapes and sizes can likewise be employed,depending, for instance, on the region or regions that a human eye wouldgenerally tend to focus on when viewing pictures or videos, such asvideo signal 110.

In an embodiment of the present invention, high scene cost detectionmodule 150 can vary the size and or shape of the central region 302. Forinstance, the size of the central region could also expand and contractwith the buffer fullness. If the buffer fullness is within limitsdetermined by comparing the buffer fullness to a buffer threshold B2, alarger central region 302 can be employed. As discussed above, the shapeof the central region can further be dynamically adjusted. In anembodiment of the present invention, the central region shape 302 coulddefault to a rectangle, but high scene cost detection module 150 couldthen eliminate macroblocks with a low variance from the central region.In another embodiment, a morphological operation could be employed toexpand or contract the boundaries of the central region 302.

FIG. 4 presents a block diagram representation of a video distributionsystem 175 in accordance with an embodiment of the present invention. Inparticular, transcoded video signal 112 is transmitted via atransmission path 122 to a video decoder 104. Video decoder 104, in turncan operate to decode the transcoded video signal 112 for display on adisplay device such as television 10, computer 20 or other displaydevice.

The transmission path 122 can include a wireless path that operates inaccordance with a wireless local area network protocol such as an 802.11protocol, a WIMAX protocol, a Bluetooth protocol, etc. Further, thetransmission path can include a wired path that operates in accordancewith a wired protocol such as a Universal Serial Bus protocol, anEthernet protocol or other high speed protocol.

FIG. 5 presents a block diagram representation of a video storage system179 in accordance with an embodiment of the present invention. Inparticular, device 11 is a set top box with built-in digital videorecorder functionality, a stand alone digital video recorder, a DVDrecorder/player or other device that stores the transcoded video signal112 for display on video display device such as television 12. Whilevideo transcoder 102 is shown as a separate device, it can further beincorporated into device 11. While these particular devices areillustrated, video storage system 179 can include a hard drive, flashmemory device, computer, DVD burner, or any other device that is capableof generating, storing, decoding and/or displaying the combined videostream 220 in accordance with the methods and systems described inconjunction with the features and functions of the present invention asdescribed herein.

FIG. 6 presents a flowchart representation of a method in accordancewith an embodiment of the present invention. In particular a method ispresented for use in conjunction with one ore more functions andfeatures described in conjunction with FIGS. 1-5. In step 500, themethod determines if a high cost scene is detected in the image. Whenthe high cost scene is detected, an enhanced quality is assigned to atleast one central region of the image as shown in step 504 and atranscoded image is generated as shown in step 506.

In an embodiment of the present invention, step 500 includes generatinga scene cost based on at least one encoding parameter of video signal.The at least one encoding parameter can include a number of bits used toencode the at least one image, an average variance of a plurality ofmacroblocks in the at least one image, and/or a quantization step value.Step 500 can further include comparing the scene cost to a scene costthreshold and detecting the high scene cost when the scene cost comparesunfavorably to the scene cost threshold.

In an embodiment of the present invention, the transcoded video signalcan be a constant bit rate signal and step 504 can assign a diminishedquality to regions of the at least one image outside of the at least onecentral region. Alternatively, the transcoded video signal can be avariable bit rate signal and step 504 can include assigning a quality toregions of the at least one image outside of the at least one centralregion that is lower than the enhanced quality. The video signal can bea MPEG-2 signal and the transcoded video signal can be an AVC signal.

FIG. 7 presents a flowchart representation of a method in accordancewith an embodiment of the present invention In particular a method ispresented for use in conjunction with one ore more functions andfeatures described in conjunction with FIGS. 1-6. In particular, themethod includes common elements of the method of FIG. 6 that arereferred to by common reference numerals. This method however, detectswhen a buffer fullness compares unfavorably to a buffer fullnessthreshold, as shown in step 502, and step 506 is based on step 504, onlywhen the buffer fullness compares favorably to the buffer fullnessthreshold.

In preferred embodiments, the various circuit components are implementedusing 0.35 micron or smaller CMOS technology. Provided however thatother circuit technologies, both integrated or non-integrated, may beused within the broad scope of the present invention.

While particular combinations of various functions and features of thepresent invention have been expressly described herein, othercombinations of these features and functions are possible that are notlimited by the particular examples disclosed herein are expresslyincorporated in within the scope of the present invention.

As one of ordinary skill in the art will appreciate, the term“substantially” or “approximately”, as may be used herein, provides anindustry-accepted tolerance to its corresponding term and/or relativitybetween items. Such an industry-accepted tolerance ranges from less thanone percent to twenty percent and corresponds to, but is not limited to,component values, integrated circuit process variations, temperaturevariations, rise and fall times, and/or thermal noise. Such relativitybetween items ranges from a difference of a few percent to magnitudedifferences. As one of ordinary skill in the art will furtherappreciate, the term “coupled”, as may be used herein, includes directcoupling and indirect coupling via another component, element, circuit,or module where, for indirect coupling, the intervening component,element, circuit, or module does not modify the information of a signalbut may adjust its current level, voltage level, and/or power level. Asone of ordinary skill in the art will also appreciate, inferred coupling(i.e., where one element is coupled to another element by inference)includes direct and indirect coupling between two elements in the samemanner as “coupled”. As one of ordinary skill in the art will furtherappreciate, the term “compares favorably”, as may be used herein,indicates that a comparison between two or more elements, items,signals, etc., provides a desired relationship. For example, when thedesired relationship is that signal 1 has a greater magnitude thansignal 2, a favorable comparison may be achieved when the magnitude ofsignal 1 is greater than that of signal 2 or when the magnitude ofsignal 2 is less than that of signal 1.

As the term module is used in the description of the various embodimentsof the present invention, a module includes a functional block that isimplemented in hardware, software, and/or firmware that performs one ormore functions such as the processing of an input signal to produce anoutput signal. As used herein, a module may contain submodules thatthemselves are modules.

Thus, there has been described herein an apparatus and method, as wellas several embodiments including a preferred embodiment, forimplementing a video transcoding coding system and a high scene costdetection module for use therewith. Various embodiments of the presentinvention herein-described have features that distinguish the presentinvention from the prior art.

It will be apparent to those skilled in the art that the disclosedinvention may be modified in numerous ways and may assume manyembodiments other than the preferred forms specifically set out anddescribed above. Accordingly, it is intended by the appended claims tocover all modifications of the invention which fall within the truespirit and scope of the invention.

1. A system for transcoding a video signal into a transcoded videosignal, the video signal including at least one image, the systemcomprising: a high scene cost detection module that detects a high scenecost corresponding to the at least one image; and an transcoder section,coupled to the high scene cost detection module, that generates thetranscoded video signal, wherein, when the high scene cost is detected,an enhanced quality is assigned to at least one central region.
 2. Thesystem of claim 1 wherein the high scene cost detection module generatesa scene cost based on at least one encoding parameter of video signal.3. The system of claim 2 wherein the at least one encoding parameterincludes a number of bits used to encode the at least one image.
 4. Thesystem of claim 2 wherein the at least one encoding parameter includesan average variance of a plurality of macroblocks in the at least oneimage.
 5. The system of claim 2 wherein the at least one encodingparameter includes a quantization step value.
 6. The system of claim 2wherein the high scene cost detection module compares the scene cost toa scene cost threshold and wherein the high scene cost detection moduledetects the high scene cost when the scene cost compares unfavorably tothe scene cost threshold.
 7. The system of claim 1 wherein thetranscoder section detects when a buffer fullness compares favorably toa buffer fullness threshold, and the encoder section assigns theenhanced quality to the central region of the at least one image onlywhen the buffer fullness compares favorably to the buffer fullnessthreshold.
 8. The system of claim 1 wherein the transcoded video signalis a constant bit rate signal and wherein the encoder section assigns adiminished quality to regions of the at least one image outside of theat least one central region.
 9. The system of claim 1 wherein thetranscoded video signal is a variable bit rate signal and wherein theencoder section assigns a quality to regions of the at least one imageoutside of the at least one central region that is lower than theenhanced quality.
 10. The system of claim 1 wherein the video signal isa MPEG-2 signal and the transcoded video signal is an AVC signal.
 11. Amethod for transcoding a video signal into a transcoded video signal,the video signal including at least one image, the method comprising:detecting a high scene cost corresponding to the at least one image; andgenerating the transcoded video signal, wherein, when the high scenecost is detected, an enhanced quality is assigned to at least onecentral region.
 12. The method of claim 11 wherein detecting the highscene cost includes generating a scene cost based on at least oneencoding parameter of video signal.
 13. The method of claim 12 whereinthe at least one encoding parameter includes a number of bits used toencode the at least one image.
 14. The method of claim 12 wherein the atleast one encoding parameter includes an average variance of a pluralityof macroblocks in the at least one image.
 15. The method of claim 12wherein the at least one encoding parameter includes a quantization stepvalue.
 16. The method of claim 12 wherein detecting the high scene costincludes comparing the scene cost to a scene cost threshold anddetecting the high scene cost when the scene cost compares unfavorablyto the scene cost threshold.
 17. The method of claim 11 furthercomprising; detecting when a buffer fullness compares favorably to abuffer fullness threshold; wherein generating the transcoded videosignal includes assigning the enhanced quality to the central region ofthe at least one image only when the buffer fullness comparesunfavorably to the buffer fullness threshold.
 18. The method of claim 11wherein the transcoded video signal is a constant bit rate signal andwherein generating the transcoded video signal assigns a diminishedquality to regions of the at least one image outside of the at least onecentral region.
 19. The method of claim 11 wherein the transcoded videosignal is a variable bit rate signal and wherein generating thetranscoded video signal assigns a quality to regions of the at least oneimage outside of the at least one central region that is lower than theenhanced quality.
 20. The method of claim 11 wherein the video signal isa MPEG-2 signal and the transcoded video signal is an AVC signal.